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Why GAO Did This Study 

Training can play a key role in 
helping the Department of 
Homeland Security (DHS) 
successfully address the challenge 
of transformation and cultural 
change and help ensure that its 
workforce possesses the 
knowledge and skills needed to 
effectively respond to current and 
future threats. This report 
discusses (1) how DHS is 
addressing or planning to address 
departmentwide training and the 
related challenges it is 
encountering; (2) examples of how 
DHS training practices, specifically 
those related to planning and 
evaluation, reflect strategic 
practices; and (3) examples of how 
DHS uses training to foster 
transformation and cultural 
change. 



What GAO Recommends 



We recommend that the Secretary 
of Homeland Security (1) adopt 
additional good planning practices, 
specifically, creating a clearer 
crosswalk between training goals 
and DHS's organizational and 
human capital strategic goals and 
developing appropriate 
performance measures and targets; 
(2) specify authority/ 
accountability relationships 
between CHCO and components on 
training; (3) ensure the department 
and components develop detailed 
training implementation plans; and 
(4) when setting funding priorities, 
give appropriate attention to 
supporting training councils and 
groups. DHS generally agreed with 
the report's recommendations. 

www.gao.gov/cgi-bin/getrpt7GAO-05-888. 

To view the full product, including the scope 
and methodology, click on the link above. 
For more information, contact George Stalcup 
at (202) 512-9490 or stalcupg@gao.gov. 
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What GAO Found 

DHS has taken several positive steps toward establishing an effective 
departmentwide approach to training, yet significant challenges remain. 

Progress made in addressing departmentwide training issues, but 
efforts are still in the early stages and face several challenges. 

Actions taken by DHS include issuing its first training strategic plan in July 
2005, establishing training councils and groups to increase communication 
across components, and directly providing training for specific 
departmentwide needs. However, several challenges may impede DHS from 
achieving its departmental training goals. First, the sharing of training 
information across components is made more difficult by the lack of 
common or compatible information management systems and a commonly 
understood training terminology. Second, authority and accountability 
relationships between the Office of the Chief Human Capital Officer and 
organizational components are not sufficiently clear. Third, DHS's planning 
may be insufficiently detailed to ensure effective and coordinated 
implementation of departmentwide training efforts. Finally, according to 
training officials, DHS lacks resources needed to implement its departmental 
training strategy. 

Examples of planning and evaluation of training demonstrate some 
elements of strategic practice. Specific training practices at both the 
component and departmental levels may provide useful models or insights to 
help others in DHS adopt a more strategic approach to training. We found 
that some components of DHS apply these practices, while others do not. 
For example, Customs and Border Protection (CBP) aligns training priorities 
with strategic goals through planning and budgeting processes. In the area of 
evaluation, the Federal Law Enforcement Training Center obtains feedback 
from both the trainee and the trainee's job supervisor to inform training 
program designers in order to make improvements to the program 
curriculum. 

Training has been used to help DHS's workforce as it undergoes 
transformation and cultural change. The creation of DHS from different 
legacy organizations, each with its own distinct culture, has resulted in 
significant cultural and transformation challenges for the department. At the 
departmental level, one of the ways DHS is addressing these challenges is by 
encouraging the transformation to a shared performance-based culture 
through the implementation of its new human capital management system, 
MAX™. DHS considers training to be critical to effectively implementing this 
initiative and defining its culture. Toward that end, the department is 
providing a wide range of training, including programs targeted to 
executives, managers, and supervisors. For example, at the component level, 
CBP has developed cross-training to equip employees with the knowledge 
needed to integrate inspection functions once carried out by three different 
types of inspectors at three separate agencies. 
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United States Government Accountability Office 
Washington, DC 20548 



September 23, 2005 

The Honorable George V. Voinovich 
Chairman 

Subcommittee on Oversight of Government Management, 
the Federal Workforce, and the District of Columbia 
Committee on Homeland Security and Governmental Affairs 
United States Senate 

Dear Mr. Chairman: 

Addressing an organization's culture — that is, its underlying assumptions, 
beliefs, values, attitudes, and expectations — is at the heart of any serious 
organizational transformation or change management initiative. As the 
Department of Homeland Security (DHS) strives to protect the nation from 
terrorism, it faces significant challenges in transforming from 22 separate 
agencies and programs to a single coordinated department, requiring the 
integration of approximately 180,000 employees as well as multiple 
management systems and processes. In recognition of these challenges, 
we have designated the implementation and transformation of the 
department as high risk. 1 

Training and development can play a key role in helping DHS successfully 
address the challenge of transformation and cultural change and help 
ensure that its workforce possesses the knowledge, skills, and 
competencies needed to effectively respond to current and future threats. 2 
In addition, a strategic approach to the management of training can help to 
effectively target limited resources and further the achievement of its 
organizational goals. To this end, our framework for assessing training 



1 GAO, High-Risk Series: An Update, GAO-05-207 (Washington, D.C.: January 2005). 

2 In previous GAO reports we have defined "training" as making available to employees 
planned and coordinated educational programs of instruction in professional, technical, or 
other fields that are or will be related to their job responsibilities. Similarly, we have 
defined "development" to generally include aspects of training, as well as structured on-the- 
job learning experiences (such as coaching, mentoring, or rotational assignments), and 
education. For the purposes of this report, "training" will be used as a shorter substitute for 
"training and development." 
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management provides a way for DHS to recognize and develop such an 
approach. 3 

You asked us to examine how DHS, as a federal agency undergoing 
transformation, uses training to help achieve its organizational goals. In 
response to your request, this report discusses (1) how DHS is addressing 
or planning to address departmentwide training and the related challenges 
it is encountering; (2) examples of how DHS training practices, specifically 
those related to planning and evaluation, reflect strategic practices; and 
(3) examples of how DHS uses training to foster transformation and 
cultural change. 

To achieve our objectives, we reviewed internal training at both the 
departmental and component levels, focusing on the systems and 
processes in place to manage, plan, and evaluate training for DHS's 
workforce. To this end, we analyzed training, management, and planning 
documents and interviewed numerous officials responsible for training 
issues in the Office of the Chief Human Capital Officer (CHCO office) and 
at six organizational components collectively responsible for training 95 
percent of the DHS workforce. To determine whether DHS used a strategic 
approach in managing, planning, and evaluating its training activities, we 
drew on our previous work regarding strategic planning and effective 
management practices, as well as criteria contained in our guide for 
assessing strategic training and development efforts in the federal 
government. We recognize that DHS provides a significant amount of 
training to external audiences, such as state and local governments; 
however, given the nature of your request and interest in examples of how 
DHS is using training to foster its organizational transformation, we did 
not include training intended for audiences external to DHS within the 
scope of our review. For more information on our scope and methodology, 
see appendix I. 

We conducted our work from November 2004 through July 2005 in 
accordance with generally accepted government auditing standards. 



RGSllltS in Brief DHS has made progress in addressing departmentwide training issues, but 

these efforts are still in the early stages and challenges may impede these 



3 GAO, Human Capital: A Guide for Assessing Strategic Training and Development 
Efforts in the Federal Government, GAO-04-546G (Washington, D.C.: March 2004). 
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efforts. Thus far, DHS has taken several steps toward establishing a 
departmentwide approach to training, including the following: 

• DHS has issued its first training strategic plan providing a strategic vision 
for departmental training. It used a collaborative and inclusive process to 
develop the plan, consulting with component training leaders and others. 

• It also has established training councils and groups with the goal of 
increasing communication across components and fostering greater 
collaboration and coordination. 

• Finally, it has provided training targeted to address specific 
departmentwide needs. Examples of areas where DHS has directly 
provided or supported training on the departmental level include 

(1) implementation of MAX™, DHS's new human capital management 
system; 4 (2) leadership development; and (3) civil rights and liberties. 

However, we identified four challenges that may impede the department 
from achieving departmental training goals. First, the sharing of training 
information across components is hampered by the lack of a common or 
compatible information management infrastructure and the absence of a 
commonly understood terminology. Officials told us that the lack of 
compatible information technology systems complicated their efforts to 
make the most efficient use of training resources across components. 
Second, authority and accountability relationships between the CHCO 
office and the organizational components are not sufficiently clear. A clear 
and agreed-upon understanding of the specific responsibilities and 
authorities of the key organizations involved in training should 
significantly improve DHS's ability to effectively implement its training 
strategies. The department recognizes this need to clarify the 
responsibilities and authorities of the CHCO office and the components 
and has addressed this need in its training strategic plan. Third, DHS's 
planning may be insufficiently detailed to ensure effective and coordinated 
implementation of departmentwide training efforts. Because they share 
authority for training, the department and the components need to develop 
detailed implementation plans to help ensure that departmentwide 
training initiatives are coordinated and effectively implemented. Fourth, 
according to training officials, DHS lacks resources needed to implement 
its departmental training strategy. 



4 According to DHS, the name MAX™ was chosen to convey the intent of the new system to 
foster "maximizing results, rewarding excellence." MAX™ covers key human capital areas, 
including pay, performance management, classification, labor relations, adverse actions, 
and employee appeals. 
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While still evolving, some of DHS's training practices at both the 
component and department levels demonstrate strategic elements in the 
areas of planning and evaluation and may provide useful models or 
insights to help others in DHS adopt a more strategic approach to training. 
We have reported previously that as part of the planning process, agencies 
demonstrating a strategic approach to training align their training efforts 
with overall organizational strategic priorities; some components of DHS 
apply these practices, while others do not. U.S. Customs and Border 
Protection (CBP), for example, employs practices intended to align 
training priorities with strategic goals through planning and budgeting 
processes. The U.S. Coast Guard also demonstrates a strategic approach 
by using a process for determining whether training is the appropriate 
intervention to address a specific performance problem. 

With respect to evaluation, we have reported that agencies demonstrating 
a strategic approach to training employ a variety of practices, such as 
systematically evaluating training, actively incorporating feedback during 
training design, and obtaining feedback from multiple perspectives. 
Several components and programs we examined at DHS demonstrate 
these practices, while others do not. For example, the Federal Law 
Enforcement Training Center (FLETC) obtains feedback from both the 
trainee and the trainee's job supervisor to inform training program 
designers making improvements to the curriculum. CBP and the CHCO 
office also use various strategies to evaluate training programs and take 
appropriate actions. 

DHS also has used training to help foster transformation and cultural 
change as the department moves from multiple distinct organizational 
cultures to a new culture that endeavors to be increasingly integrated and 
performance focused. This is not an easy process, and the creation of DHS 
from 22 different agencies and programs has resulted in considerable 
cultural and transformation challenges. At the department level, DHS has 
addressed these challenges by encouraging the transformation to a shared 
performance-based culture through the adoption of a new human capital 
management system, known as MAX™. As an essential part of 
implementing this initiative, DHS developed targeted training for 
executives, managers, and supervisors, providing these groups with the 
tools and information needed to champion the benefits of a performance- 
based culture and successfully implement MAX™ in their components. In 
another example, this time at the component level, in order to improve 
coordination and communication across inspection functions and enhance 
flexibility of the workforce, CBP created the new positions of CBP officer 
and CBP agriculture specialist. Cross-training of employees in these new 
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positions helped CBP to integrate the inspection functions of three former 
agencies. In addition, CBP designed and piloted a training module that 
specifically targeted how supervisors could more effectively understand 
the value and perspective of staff coming from legacy organizational 
cultures. 

To help DHS further establish and implement an effective and strategic 
approach to departmental training, we recommend that the Secretary of 
Homeland Security take the following actions: (1) adopt additional good 
strategic planning and management practices to enhance the department's 
training strategic plan by creating a clearer crosswalk between specific 
training goals and objectives and DHS's organizational and human capital 
strategic goals and developing appropriate performance measures and 
targets; (2) clearly specify authority and accountability relationships 
between the CHCO office and organizational components regarding 
training, as a first step toward addressing issues DHS has identified for 
fiscal year 2006; (3) ensure that the department and component 
organizations develop detailed implementation plans and related 
processes for training initiatives; and (4) when setting funding priorities, 
give appropriate attention to providing resources to support training 
councils and groups to further DHS's capacity to achieve its 
departmentwide training goals. 

We provided a draft of this report to the Secretary of Homeland Security 
for comment. DHS generally agreed with our recommendations. The 
department provided technical comments that we incorporated where 
appropriate. DHS's written response is reprinted in appendix III. 



BcLCkSrOlind. ^ e Homeland Security Act of 2002 created DHS, bringing together 22 

" agencies and programs responsible for important aspects of homeland 

security. 5 The intent behind the creation of a single department was to 
improve coordination, communication, and information sharing among 
these previously separate entities, thereby increasing their effectiveness in 
protecting the nation's security. Each of these organizations brought with 
it the capacity and expertise to provide training for its particular aspect of 
homeland security. For example, in several cases such as the Coast Guard 
and FLETC, this training capacity, as well as the management systems 
supporting it was transferred intact with the creation of the new 



5 Homeland Security Act of 2002, Pub. L. No. 107-296 (Nov. 25, 2002). 
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department. In other cases, such as CBP and U.S. Immigration and 
Customs Enforcement (ICE), the training functions of legacy organizations 
were merged. Table 1 presents information on selected training 
characteristics of components in our review, including the origin of each 
component's training function. In addition, the Act led to the creation of 
the CHCO position in DHS responsible for, among other human capital 
topics, oversight and planning of the training of employees. 6 The CHCO, 
who reports directly to the department's Under Secretary for Management, 
has primary responsibility for defining and developing the department's 
role regarding training. Figure 1 depicts these positions as well as the 
department's major components in the context of DHS's overall 
organizational structure. 

Training both new and current staff to fill new roles and work in different 
ways will play a crucial part in the ability of federal departments and 
agencies, such as DHS, as they work to successfully transform their 
organizations. In 2004, we issued an assessment guide that introduces a 
framework for evaluating the management of training in the federal 
government. 7 As presented in our guide, the training process can be 
segmented into four broad, interrelated phases: (1) planning/front-end 
analysis, (2) design/development, (3) implementation, and (4) evaluation. 
For each of these phases, we summarize key attributes of effective training 
programs and offer related issues and questions. Using this framework, 
this report identifies selected strategic training practices, with a focus on 
the planning and evaluation phases, that may offer an opportunity for 
others in DHS to build on experiences and practices discussed below. 



This section of the Homeland Security Act of 2002, also known as the Chief Human 
Capital Officers Act of 2002, led to the creation of chief human capital officers in each 
federal department. Pub. L. No. 107-296, §§ 1301-1305 (Nov. 25, 2002). 

7 GAO-04-546G. 
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Table 1 : Responsibilities and Selected Training Characteristics of DHS Components Included in Our Review 



U.S. Citizenship and 
Immigration Services 
(CIS) 

10,207 FTEs 



U.S. Coast Guard 

46,847 FTEs 



U.S. Customs and Border 
Protection (CBP) 

40,761 FTEs 



Overview of component responsibilities 

• Administer immigration and naturalization adjudication functions, including immigrant visa petitions, 
naturalization petitions, and asylum and refugee applications 

• Establish and administer policies for immigrant services and benefits 
Selected characteristics of training 

• Training function was transferred from former Immigration and Naturalization Service with the 
exception of law enforcement and soft skills training 

• Conducts extensive field training through district offices and service centers 

• Operates CIS Academy at FLETC site in Glynco, GA, for basic training of new employees 

• Provides refresher training for adjudicators at field sites 

• Supervisory training provided by ICE's Leadership Development Center in Dallas, TX, through a 
shared services agreement between CIS, ICE, and CBP 

Overview of component responsibilities 

• Protect the public, environment, and U.S. economic interests in nation's ports, waterways, coasts, and 
international waters 

• Specific responsibilities include: Maritime safety (e.g., search and rescue), maritime mobility (e.g., 
aids to navigation and waterways management), protection of natural resources, maritime security 
(e.g., drug interdiction), and national defense 

Selected characteristics of training 

• Training function transferred as a whole along with the rest of the Coast Guard from Department of 
Transportation 

• Operates multiple training programs for (1) indoctrination for new employees, (2) apprenticeship after 
indoctrination course is completed, and (3) specialized skills (e.g., law enforcement inspections) 

• Operates special leadership training program at Leadership Development Center, New London, CT 

• Uses some Department of Defense training courses for specialized skills 
Overview of component responsibilities 

• Protect U.S. borders from terrorism, at and between official ports of entry, and foster legitimate trade 
and travel 

Selected characteristics of training 

• Training function structure and processes largely carried over from former U.S. Customs organization 

• Basic CBP officer training provided at CBP Academy at FLETC site in Glynco, GA, and basic CBP 
agricultural specialist training at CBP Academy at U.S. Department of Agriculture-Professional 
Development Center in Frederick, MD 

• Provides basic Border Patrol training at Border Patrol Academy in Artesia, NM 

• Provides extensive field training at ports throughout the United States 

• Supervisory training provided at ICE's Leadership Development Center in Dallas, TX, through a 
shared services agreement between CIS, ICE, and CBP 
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Federal Law Enforcement 
Training Center (FLETC) 

959 FTEs 



U.S. Immigration and 
Customs Enforcement 
(ICE) 

14,950 FTEs 



U.S. Secret Service 

6,526 FTEs 



Transportation Security 
Administration (TSA) 

52,467 FTEs 



Overview of component responsibilities 

• Train and prepare law enforcement professionals across government 
Selected characteristics of training 

• Training function transferred as a whole from Department of the Treasury with formation of DHS 

• Consolidates law enforcement training at five FLETC academies for multiple DHS components 

• FLETC budget covers most of the basic training provided to DHS components; auxiliary costs, such 
as room and board, are reimbursed by components 

Overview of component responsibilities 

• Prevent acts of terrorism by targeting people, money, and materials that support terrorist and criminal 
activities focusing on the nation's border, economic, transportation, and infrastructure security 

Selected characteristics of training 

• Training function partly carried over from legacy Immigration and Naturalization Service and legacy 
Customs Inspections and includes a separate training organization for the Federal Protective Service 

• Federal Protective Service trains its uniformed officers at FLETC site in Glynco, GA, and ICE 
Academy is located at same FLETC site 

• ICE's Leadership Development Center in Dallas, TX, provides supervisor and manager training for 
multiple DHS components 

Overview of component responsibilities 

• Protect the President and other designated personnel 

• Protect the country's currency and financial infrastructure 

• Provide security for designated national events 
Selected characteristics of training 

• Training function transferred as a whole along with the rest of the Secret Service from Department of 
the Treasury 

• Trains both special agents and uniformed law enforcement officers with most instruction taking place 
at Rowley Training Center in Beltsville, MD 

• Basic training for new employees takes place at FLETC site in Glynco, GA 
Overview of component responsibilities 

• Protect the nation's transportation systems to ensure freedom of movement for people and commerce 
Selected characteristics of training 

• Training function greatly expanded after transfer from Department of Transportation with formation of 
DHS; separate training organization in place for the Federal Air Marshal Service, which was recently 
transferred as a result of DHS's Second Stage Review 

• Some training delivered through TSA headquarters office in Arlington, VA, which also oversees 
contracted training activities through its quality assurance unit 

• Most training delivered in field (airport) sites through contractors and approved instructors 

• Operates an academy in Artesia, NM, and an academy at the FLETC site in Glynco, GA 

• Federal Air Marshal Service trains new hires at an initial program in Artesia, NM, and then they 
receive specialized training in Atlantic City, NJ; Federal Air Marshal Service field offices also conduct 
ongoing training 

Sources: GAO presentation of DHS information and the President's Fiscal Year 2006 Budget. 

Note: Figures showing full-time equivalents (FTE) for components reflect FTEs listed under "current 
services" from the President's Fiscal Year 2006 Budget. 
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The results of a governmentwide survey conducted by the Office of 
Personnel Management in 2004 on human capital practices and employee 
attitudes suggest that efforts to identify and build upon examples of good 
training practice within DHS may be particularly relevant. 8 For each of the 
eight questions in the 2004 Federal Human Capital Survey that focused on 
training related topics, the percentage of DHS respondents providing 
positive responses (typically the top two options on a five-point scale) was 
lower than the governmentwide average. In fact, the DHS response ranged 
from 5 to 20 percentage points lower than the governmentwide average for 
the same questions. For example, 54 percent of respondents at DHS 
indicated that they received the training they needed in order to perform 
their jobs, compared to 60 percent governmentwide. Half (50 percent) of 
DHS respondents said that they were either satisfied or very satisfied with 
the training they received for their present jobs, as opposed to 55 percent 
that expressed these levels of satisfaction governmentwide. The largest 
difference involved having electronic access to learning and training 
programs, where 51 percent of DHS respondents responded positively, 
compared to 71 percent governmentwide. A DHS official told us that the 
department is aware of the challenges reflected in these data and is 
currently exploring options with the Office of Personnel Management to 
conduct further analysis. The aim of this work would be to identify areas 
where DHS might target additional attention as well as provide a baseline 
for future attitude measures. 



According to the Office of Personnel Management, both the survey's governmentwide 
findings and DHS-specific findings are generalizable to their respective populations. For 
additional findings from the 2004 Federal Human Capital Survey on these and other 
training and non-training-related questions, see www.fhcs2004.opm.gov. 
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Figure 1 : DHS Organizational Structure 
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Source: GAO presentation of DHS information. 

Note: This organization chart shows the expected end state resulting from a reorganization 
announced by the department in July 2005 at the conclusion of DHS's Second Stage Review 
process. 
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DHS has made progress in addressing departmentwide training issues and 
these efforts reflect some of the elements of a strategic approach toward 
training as described in our previous work. 9 Most training-related activities 
at DHS — such as planning, delivery, and evaluation — primarily take place 
at the component level and relate to mission issues. Therefore, any 
successful approach regarding departmentwide training issues will require 
the concerted and coordinated efforts of multiple components within DHS 
as well as the ability of the CHCO to effectively lead a network of different 
training organizations. The department's current efforts, although 
promising, are still in the early stages and they face significant challenges. 
Unless these challenges are successfully addressed they may impede 
DHS's ability to achieve its departmentwide training goals. 



DHS Recently Developed a DHS recently developed a coordinated departmental training strategy that 
Departmental Training supports broader human capital and organizational goals and objectives. 

Strategy ^ e nave previously reported that effective organizations establish clear 

goals with an authority structure able to carry out strategies and tactics, 
that is, the day-to-day activities needed to support the organization's vision 
and mission. By so doing, a well-designed training function can be directly 
linked to the organization's strategic goals and help to ensure that the 
skills and competencies of its workforce enable the organization to 
perform its mission effectively. 

DHS's department-level training strategy is presented in its human capital 
and training strategic plans. Issued in October 2004, its human capital 
strategic plan includes selected training strategies, such as developing a 
leadership curriculum to ensure consistency of organizational values 
across the department and using training to support the implementation of 
the new DHS human capital management system, MAX™. In July 2005, 
DHS issued its first departmental training plan, Department of Homeland 
Security Learning and Development Strategic Plan, which provides a 
strategic vision for departmentwide training. This plan is a significant and 
positive step toward addressing departmentwide training challenges. The 
plan identifies four short-term goals for fiscal year 2006 and one long-term 
goal for fiscal years 2006 through 2010. Among the short-term goals are 
such tasks as defining the scope of training activities and improving the 
governance process between the CHCO office and individual 
organizational components, supporting the rollout of MAX™, 



DHS Has Made 
Progress in 
Addressing 
Departmentwide 
Training Issues, but 
Challenges May 
Impede Its Efforts 
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identifying/implementing best practices, and addressing specific concerns 
regarding DHS's training facilities and advanced distributed learning 
studies. The plan also articulates a long-term goal for DHS to "become a 
recognized world-class learning organization where managers and 
supervisors effectively lead people." 

Each of these goals is followed by supporting strategies and tactics. For 
example, to achieve its goal of ensuring the best use of training resources 
through the identification and implementation of best practices, the plan 
identifies specific strategies, one of which is to improve the awareness of 
ongoing DHS training activities among organizational components. This 
strategy is, in turn, supported by still more specific tactics such as 
developing a site on the DHS Interactive system to facilitate the sharing of 
information across the training community. 

More significant than the fact that DHS issued a training strategic plan 
document is the fact that DHS followed an inclusive and collaborative 
process while developing it. We have previously reported that for high- 
performing, results-oriented organizations, a strategic plan is not simply a 
paper-driven exercise or onetime event, but rather the result of a dynamic 
and inclusive process wherein key stakeholders are consulted and 
involved in the identification of priorities and the formation of strategies. 10 
When creating its plan, DHS consulted training leaders at components 
throughout the department, in addition to others, to help develop and 
review its content. Several training leaders we spoke with thought highly 
of this process and the extent to which it provided them opportunities to 
contribute and comment on the draft plan. 

DHS has made considerable progress in addressing departmentwide 
training issues through the development of its first training strategic plan. 
However, there are areas where future efforts can be improved. 

Linkage to DHS organizational and human capital strategic plans. 

Our past work on strategic planning and management practices shows that 
effective strategic plans describe the alignment between an agency's long- 
term goals and objectives and the specific strategies planned to achieve 
them. Clearly linking training tactics with particular organizational 
objectives creates a direct line of sight that can both facilitate the ability of 



GAO, Executive Guide: Effectively Implementing the Government Performance and 
Results Act, GAO/GGD-96-118 (Washington, D.C: June 1996). 
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staff to work toward mission goals and enable stakeholders to provide 
meaningful oversight. In the introduction to the DHS training strategic 
plan, the department's CHCO highlights the value of this practice stating 
that "the key purpose of [the] plan is to align our education, training and 
professional development efforts with the President's Management 
Agenda and the Department's vision, mission, core values and strategic 
plan." 

The DHS training strategic plan contains examples of goals, strategies, and 
tactics that align with and support goals found in the department's human 
capital and organizational strategic plans; however, these linkages are 
never actually identified or discussed in the plan itself. For example, the 
DHS training strategic plan contains a goal and several tactics related to 
MAX™ training. These, in turn, support a MAX™ goal and strategy in the 
department's human capital strategic plan as well as the "organizational 
excellence" goal of the DHS strategic plan. However, the training strategic 
plan does not show these linkages. Identifying such linkages, either in the 
training plan itself or in an appendix, would more clearly communicate to 
both internal and external stakeholders the connections and justifications 
for specific training goals, strategies, and tactics. 

DHS's own human capital strategic plan provides an illustration of one 
way to communicate linkages between goals and strategies contained in 
the plan and the broader organizational goals they are intended to support. 
For example, in an appendix, the DHS human capital strategic plan 
contains a matrix that directly links strategies, such as developing a new 
Senior Executive Service (SES) performance management system, with 
specific objectives contained in the DHS strategic plan as well as the 
President's Management Agenda human capital standards for success. 

Usefulness of performance measures. We have previously reported 
several key characteristics of effective strategic and management plans, 
including the need for performance measures. 11 Appropriate performance 
measures along with accompanying targets are important tools to enable 
internal and external stakeholders to effectively track the progress the 
department is making toward achieving its training goals and objectives. 
To this end, organizations may use a variety of performance measures — 



GAO, Managing for Results: Next Steps to Improve the Federal Government's 
Management and Performance, GAO-02-439T (Washington, D.C.: Feb. 15, 2002). 
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output, efficiency, customer service, quality, and outcome — each of which 
focuses on a different aspect of performance. 

The DHS training strategic plan contains few specific performance 
measures for its goals or strategies and all of these are output measures. 
For example, the plan makes use of output measures in its requirement 
that certain actions, such as the development of a new management 
directive or the chartering of a team, be completed by the end of fiscal 
year 2006, and in establishing a deadline for when reports need to be 
completed in order to be included in the 2007 plan. In contrast to output 
measures like these, which gauge the level of activity or effort by 
measuring whether a particular thing is produced or service performed, 
other types of measures, such as measures of customer satisfaction or 
program outcomes, focus on the impact or results of activities. By 
appropriately broadening the mix of measures it uses and more clearly 
identifying targets against which DHS can assess its performance, DHS 
can improve the usefulness of its plan. After we completed our audit work, 
DHS training officials informed us that they decided to delay the 
development of performance measures until the rollout of the plan, when 
they could be developed by individual teams, as needed. They 
subsequently informed us that these teams will be held accountable to 
establish further performance measures that are outcome based and 
results oriented. 

DHS's human capital strategic plan again provides an illustration of how 
the department's training strategic plan might begin to work toward the 
inclusion of different types of performance measures. For example, 
accompanying the strategy that DHS assess the feasibility of establishing a 
21st Century Leadership Training and Development Institute, the plan 
identifies two performance measures — customer satisfaction and cost of 
delivery — along with specific targets for each. For the customer 
satisfaction measure, the plan establishes a target of 4.5 on a scale from 1 
to 5. The plan also includes specific tactics to achieve the strategy, such as 
developing and obtaining cross-organizational support, developing 
measures and methodologies for leadership training, and implementing a 
learning management system, along with key milestone dates for 
completing them. 

The department may benefit from considering the experiences of leading 
organizations regarding the development of results-oriented performance 
measurement. In general, results-oriented organizations we have studied 
that were successful in measuring their performance developed measures 
that were 
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• tied to program goals and demonstrated the degree to which the desired 
results were achieved, 

• limited to the vital few that were considered essential to producing data 
for decision making, 

• responsive to multiple priorities, and 

• responsibility linked to establish accountability for results. 12 



Similar to the consultative process DHS followed when developing the 
goals and strategies contained in its training strategic plan, decisions 
concerning the selection of an appropriate set of performance measures 
should also be based on input from key stakeholders to determine what is 
important to them when assessing the department's performance 
regarding training. Clear and appropriate performance measures, 
developed in this way, can also provide DHS with valuable information, 
especially significant in the current fiscal environment, when it seeks to 
justify requests for resources from Congress. 



Training Councils and 
Groups Created by DHS 
Increase Communication 
across Components 



Under the overall direction of the CHCO office, DHS has established a 
structure of training councils and groups that cover a wide range of issues 
and include representatives from each organizational component within 
DHS. The department is in the process of using these bodies to facilitate 
communication and the sharing of information within its diverse training 
community. In some instances, these councils and groups foster greater 
collaboration and coordination on training policies, programs, and the 
sharing of training opportunities. We have previously reported that 
agencies with a strategic approach to training recognize the importance of 
having training officials and other human capital professionals work in 
partnership with other agency leaders and stakeholders on training 
efforts. 13 



The Training Leaders Council (TLC) plays a vital role in DHS's efforts to 
foster communication and interchange among the department's various 
training communities. This council consists of senior training leaders from 
each of the department's components as well as representatives from 
several department-level headquarters staff and support organizations 
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with an interest in training-related issues. 14 Started in October 2004 and 
formally chartered by the CHCO in March 2005, the TLC's mission is to 
establish and sustain a collaborative community with the aim of promoting 
high-quality training, education, and development throughout DHS. To this 
end, it functions as a convener of training leaders from throughout the 
department and provides an overarching framework for several 
preexisting training groups and councils that were reestablished as 
standing committees of the TLC. Membership of the TLC consists of senior 
training leaders from each DHS component. In addition, most of these 
leaders as well as other training staff serve on one or more of its 
subgroups. See figure 2 for descriptions of the TLC and each of its 
subgroups. 



The TLC includes representatives from the following entities: CBP, CIS, FLETC, ICE, 
TSA, Emergency Preparedness and Response/Federal Emergency Management Agency, 
Information Analysis and Infrastructure Protection, Science and Technology, the Coast 
Guard, the Secret Service, and US-VISIT. The TLC also includes representatives from the 
following DHS department level organizations: CHCO office, Office of the Chief 
Information Officer, Office of the Inspector General, Office of the General Counsel, Office 
of Civil Rights and Civil Liberties, Office of Counter Narcotics, and Office of State and 
Local Coordination and Preparedness/Center for Domestic Preparedness. 
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Figure 2: Departmental Training Councils and Subgroups at DHS 
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One key function of the TLC and these other training groups is to serve as 
a "community of practice" wherein officials can discuss common training 
challenges and share knowledge and best practices. For example, the 
Training Evaluation and Quality Assurance Group, composed of DHS 
training professionals responsible for evaluating and ensuring the quality 
of DHS training programs, conducted an informal survey of evaluation 
practices in various components with the intent of identifying effective 
evaluation approaches. A training official involved in the group told us that 
this survey was particularly important for the department's newer 
organizations, such as the Directorate for Information Analysis and 
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Infrastructure Protection, which need to establish new practices from 
scratch. According to this official, his directorate and other organizations 
within DHS plan to use the group as a way to tap into the experience of 
other components within the department, such as CBP and FLETC, which 
have considerably more experience with training evaluation. 

In addition to sharing information about training practices, these groups 
can also provide a forum for exchanging practical information with the 
goal of making more efficient use of existing resources. For example, one 
training official told us that as a result of information obtained at TLC 
meetings, the official became aware of the existence of free training space 
available at facilities of two other components located in the Washington, 
D.C., metro area. Also, as a result of participating in these meetings, the 
official's organization was able to send an additional person to the Federal 
Executive Institute after becoming aware that another component had 
surplus spaces and was offering them at a reduced price to other 
components within DHS. 15 

Another role carried out by the TLC is to collaborate on the formulation of 
training policies and advise the department's CHCO accordingly. For 
example, the TLC, in cooperation with staff from the CHCO office and an 
external contractor, conducted a survey of training sites throughout the 
department in 2004. This study cataloged available physical resources and 
site capacities with the aim of identifying potential opportunities to share 
these resources more efficiently, consolidate unneeded or duplicative 
sites, and identify other opportunities to increase training collaboration 
and effectiveness. This effort resulted in a series of recommendations that 
were subsequently incorporated into the department's training strategic 
plan. 

The activities of the department's Advanced Distributed Learning Group 
(ADLG) provides another example of how training officials from different 
components have worked together to develop proposals for solutions to 
departmentwide challenges. This group identified several issues in the 



15 The Government Employees Training Act provides that an agency can extend its training 
courses to employees of other government agencies (5 U.S.C. § 4104(2)). A Comptroller 
General decision reviewed the legislative history of this provision and concluded that 
training can be provided on a reimbursable or a nonreimbursable basis, at the discretion of 
the agency offering the training (B-193293, Nov. 13, 1978). A DHS component that is 
offering training is authorized to make its courses available to other employees of DHS or 
of other government agencies. 
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area of technology and learning, including the need for a compatible IT 
infrastructure across components and the fact that some components 
lacked established systems with which to coordinate and manage training 
opportunities and attendance. Working with an outside consultant, the 
ADLG's efforts resulted in a proposal that DHS create a new Advanced 
Distributed Learning (ADL) Program Management Office to oversee the 
process of setting common standards. This proposal was subsequently 
included as part of the department's training strategic plan. In addition, the 
ADLG's work also led to DHS entering into a memorandum of 
understanding with the Office of Management and Budget and the Office 
of Personnel Management to create a DHS headquarters learning 
management system. Throughout this process, the ADLG represented the 
interests of the DHS training community as it worked with representatives 
from the Chief Information Officer's office and other functions within the 
department, as well as outside consultants. 

Despite these positive steps, DHS's effort to foster communication and 
coordination through departmentwide training councils and groups is at a 
relatively early stage and so far has produced varied results. Some training 
organizations, such as the TLC and ADLG, have met regularly leading to 
tangible results, while others such as the Training Evaluation and Quality 
Assurance Group, have met a few times and have only begun to set the 
groundwork for substantive coordination and collaboration in these areas. 
In addition, a training official told us that even active organizations like the 
TLC have encountered difficulties related to the relative lack of staff 
support for these efforts. As a result, additional burdens sometimes fall to 
the leaders and members of these groups who, in addition to serving on 
one or more departmental training groups or councils, must carry out full- 
time training positions at their home components. 



DHS Provides Training to 
Address Selected 
Departmentwide Needs 



Another way DHS addresses departmentwide training issues is to directly 
provide training interventions or resources that address selected 
departmentwide needs, goals, or objectives. Three examples of the areas 
where DHS has worked to directly provide or support training on the 
departmental level are the following: (1) training related to the 
implementation of MAX™, (2) DHS leadership development, and 
(3) training related to civil rights and civil liberties. 

Training for MAX™ implementation. DHS's new human capital 
management system, known as MAX™, represents a fundamental change 
in many of the department's human capital policies and procedures that 
will affect a large majority — approximately 110,000 — of its civilian 



Page 19 



GAO-05-888 Training at DHS 



employees. MAX covers many key human capital areas, such as pay, 
performance management, classification, labor relations, adverse actions, 
and employee appeals, and will be implemented in phases affecting 
increasing numbers of employees over the next several years. 

DHS correctly recognizes that a substantial investment in training is a key 
aspect of effectively implementing MAX™, and in particular, the new 
performance management system it establishes. The need for in-depth 
performance management and employee development training is further 
supported by the department's results on the 2004 Federal Human Capital 
Survey. In this survey, just over half of DHS respondents — 51 percent — 
believe supervisors or team leaders in their work units encourage their 
development at work, significantly less than the governmentwide response 
of 64 percent. DHS officials said they plan to educate all affected DHS 
employees on the details of the new system, how it will affect them, and 
the purpose of the changes. To do this, the department decided to develop, 
coordinate, and manage MAX™ training centrally through the CHCO office 
and offered its first training in May 2005. DHS plans to continue to provide 
its workforce with MAX™ training over the next several years following a 
phased approach that takes into account both when individual provisions 
of the new regulations take effect as well as the different audiences that 
exist within the DHS community, including human capital personnel, 
supervisors, and general employees. See figure 3 for a depiction of planned 
training during 2005 and its intended audiences. 
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Figure 3: Timeline for MAX HR Training Scheduled through the End of 2005 
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Source: GAO representation of DHS information. 

Note: This training schedule reflects the department's plans as of early August 2005. However, we 
were told by a DHS official that these dates are subject to change. 



The department has worked with contractors to develop training that uses 
a variety of approaches, including classroom instruction, ADL, handbooks, 
manuals, and quick reference guides, depending on specific needs. For 
example, in May 2005, labor relations/employee relations specialists and 
attorneys in the department received 2-Vi days of training on the 
provisions of the new regulations and the major difference between them 
and previous programs. Structured as a "train the trainer" type 
intervention intended to prepare participants to conduct supervisor 
briefings in their own components, this was an instructor-led course held 
at sites across the country. In addition to educating individuals about the 
regulations, procedures, and systems associated with MAX™ and the 
adoption of a new performance management system, the department also 
plans to offer training specifically targeted to developing the skills and 
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behaviors that will be necessary for its successful implementation. For 
example, in July 2005 supervisors began to receive training on techniques 
for providing meaningful feedback to, coaching, and mentoring employees. 

DHS leadership development training. Leadership development is 
another area top management in DHS acknowledged as appropriate for 
departmentwide training to supplement existing component-level 
offerings. In 2004, the Secretary of Homeland Security announced the 
"One DHS" policy that identified the need to establish a common 
leadership competency framework for the department, as well as a unified 
training curriculum for current and future leaders. The purpose of this 
framework was to identify the skills, abilities, and attributes necessary for 
success as a DHS leader and to establish measurable standards for 
evaluation. 

To this end, the CHCO established the DHS Leadership Training and 
Development Group (LTDG), comprising training officials from each DHS 
component who combined an expertise in leadership development with 
personal knowledge of the missions and unique aspects of their particular 
organizational components. The LTDG met regularly from late 2003 to 
mid-2004. During this time, the group developed a set of new core 
leadership competencies for DHS supervisors, managers, and executives, 
which it issued in April 2004. According to a DHS official, since the 
development of these new competencies they have been used by one 
component as part of its own leadership development plan and they have 
also helped to guide and inform current MAX™ leadership development 
efforts. 

DHS has recently taken steps regarding another facet of its leadership 
development initiative — its SES Candidate Development Program. In June 
2005, DHS issued a management directive establishing the SES Candidate 
Development Program, which included a rigorous selection process and 
critical leadership development opportunities, such as mentoring, 
developmental assignments, and action learning designed to give SES 
candidates experience in different job roles. DHS initially announced that 
it planned to implement the program in fiscal year 2005, but now may 
delay doing so until fiscal year 2006. 

Civil rights/civil liberties training. A third area in which DHS has taken 
steps to provide or support departmental training involves civil rights and 
civil liberties. FLETC's Behavioral Science Division and Legal Division, 
working with the DHS Office of Civil Rights and Civil Liberties, produced 
several training interventions, including Web-based, CD-ROM, and in- 



Page 22 



GAO-05-888 Training at DHS 



person programs designed to increase sensitivity and understanding in 
protecting human and constitutional rights. As part of this effort, FLETC 
held diversity seminars that focused on promoting understanding and 
respect of religious practices, particularly involving those of the Arab and 
Muslim communities. In another example of this effort, the Office of Civil 
Rights and Civil Liberties produced Web-based training on current policies 
regarding racial profiling. 

Our interviews with DHS training leaders suggest that further 
improvements can be made in communicating the availability of selected 
departmentwide training programs and resources. Staff at the Office of 
Civil Rights and Civil Liberties provided copies of its civil rights and 
liberties programs to training offices at each component in the 
department. While some senior training officials told us that their 
components actively disseminated this material by placing it on the 
component's training Web site or incorporating it into preexisting courses, 
other senior training officials we spoke with were unaware of any 
departmental training on these topics. In addition, other officials told us 
that their component's training office had independently developed its 
own material on Arabic sensitivity training, wholly apart from similar 
efforts undertaken by others in the department. More specifically, they 
told us that their development of certain training modules predated the 
development of very similar modules later prepared by DHS's Office of 
Civil Rights and Civil Liberties and FLETC, leading these officials to 
conclude that they may have been able to assist departmental efforts by 
sharing their work had they been aware of them. 



Challenges May Impede 
Achievement of 
Departmentwide Training 
Goals 



Sharing of Training Information 
across Components Made More 
Difficult by Lack of Compatible 
Infrastructure and Commonly 
Understood Terminology 



As DHS moves forward, it faces challenges to achieving departmentwide 
training goals. These challenges include lack of common management 
information systems, the absence of commonly understood training 
terminology across components, the lack of specificity in authority and 
accountability relationships between the CHCO office and components, 
insufficient planning for effective implementation, and insufficient 
resources for ensuring effective implementation of training strategies. 

The formation of DHS from 22 legacy agencies and programs has created 
challenges to achieving departmentwide training goals. Of particular 
concern to the training officials we spoke with are the lack of common 
management information systems and the absence of commonly 
understood training terminology across components. The training 
functions at DHS's components largely operate as they did before the 
creation of the department, with many of the same policies, practices, and 
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infrastructures of their former organizations, and within these 
organizations are, for the most part, the same training leaders. It will take 
time for these organizations to evolve into a coordinated, integrated 
department. We have previously reported that successful transformations 
of large organizations, even those faced with less strenuous 
reorganizations and pressure for immediate results than DHS, can take 
from 5 to 7 years to fully take hold. 16 

One issue DHS officials raised was the lack of common or compatible 
management information systems, such as information technology or 
financial management, which can inform decision makers' efforts to make 
efficient use of training resources across components. For example, DHS 
officials stated that a key challenge they encountered involved the 
difficulty of knowing what others were doing outside their particular 
offices or components. DHS lacks any unified sourcebook that employees 
could consult for the names, telephone numbers, and other relevant 
information of key contact persons in areas such as acquisition. Obtaining 
accurate information about resources and products available in the 
marketplace as well as data on users, vendors, and kinds of work has been 
a challenge to that effort. Another issue cited by officials concerned the 
lack of compatibility between learning management systems across 
components. In addition, some training officials expressed concerns about 
the accuracy or timeliness of some training data, which can limit or at 
least considerably delay their ability to track and fully account for funds 
spent on training and training-related travel. DHS has several efforts under 
way to address these issues, including the development of an online 
training facilities inventory intended to increase awareness of existing 
resources across the department and its decision to begin developing 
common ADL policies and standards. 

Officials also told us that there was little or no common understanding 
among DHS organizational components regarding the meaning of such 
basic terms as "subject matter expert," "orientation," and even "training." 
The lack of commonly understood terminology has presented challenges 
when officials from different components, including those participating in 
departmental training councils and groups, try to share practices with 
each other. These officials told us that the lack of commonly understood 
terminology can also affect their interactions with outside entities, such as 



GAO, Results-Oriented Cultures: Implementation Steps to Assist Mergers and 
Organizational Transformations, GAO-03-669 (Washington, D.C.: July 2, 2003). 
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contractors and state and local agencies. Besides facilitating 
communications and enabling components to share practices, a DHS 
official told us that a common nomenclature would increase the 
transparency of training practices to external contractors as well as the 
internal DHS training community. The department's training strategic plan 
calls for the creation of a common training language and glossary of terms 
in fiscal year 2006, and officials told us that they are currently in the early 
stages of creating such a glossary. 

Authority and Accountability An effective management control environment appropriately assigns 
Relationships between CHCO authority and delegates responsibility to the proper personnel to achieve 
Office and Components Are organizational goals and objectives. 17 In such an environment, staff 

Not Sufficiently Specific members who are delegated responsibility are given corresponding 

authority. In light of this, DHS's adoption of a "dual accountability" 
governance structure in 2004 presents certain challenges. Under this 
concept, heads of organizational components and the CHCO share 
responsibility for effective training in DHS. With a shared responsibility for 
DHS training, both the CHCO and component heads should have 
appropriate authority for making decisions regarding training. 18 DHS does 
not specify how authority for training matters will be shared between the 
CHCO office and components for budgeting, staffing, and policy (e.g., 
determining which training functions, if any, should remain with 
components or be performed by DHS headquarters). The DHS 
management directive on training currently in place is a high-level two- 
page document that provides very few specifics on policies, procedures, 
and authorities for the CHCO office and the components. 

The department recognizes the need to clarify the responsibilities and 
authorities of the CHCO office and the components, as indicated by its 
inclusion in the DHS training strategic plan. Many of the tactics included in 
the plan would be difficult to successfully implement without first having a 



' GAO, Internal Control: Standards for Internal Control in the Federal Government, 
GAO/AIMD-00-21.3.1 (Washington, D.C.: November 1999). 

18 The DHS management directive entitled "Human Capital Line of Business Integration and 
Management" specifies several roles and responsibilities for the CHCO and component 
heads. Roles for the CHCO include advising and assisting top DHS officials on training 
issues and designing processes and systems to achieve departmentwide training goals. For 
example, component heads have the role of recruiting, hiring, and reviewing human capital 
officials, including training staff. However, DHS has not yet specified, in detail, the 
responsibility, accountability, and authority of the CHCO and the component heads, 
although officials have indicated that they plan to do so during fiscal year 2006. 
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clear understanding of the responsibilities and authorities of the key 
organizations involved. More specifically, in the absence of clear authority 
relationships, decisions regarding how particular component training 
goals and strategies are to be incorporated in the DHS training strategic 
plan, or which training facilities should be consolidated to achieve 
departmental efficiencies, will be difficult to make. Without moving ahead 
with this effort in a timely fashion and completing the process of 
specifying how the CHCO office and components will share authority over 
training matters, it will be difficult for DHS to make the progress 
necessary on its departmentwide training agenda if it is to effectively 
implement the many strategies and tactics planned for fiscal year 2006. 

In addition, DHS's efforts at coordinating training across components and 
clarifying roles and relationships between departmental functions and 
organizational components may be further hampered by the fact that the 
management directive governing the integration of the human capital 
function claims that the Coast Guard 19 and the Secret Service 20 are 
statutorily exempt from its application. We found no reasonable basis to 
conclude that the directive could not be made applicable to them and are 
not aware of any explicit statutory exemption that would prevent the 
application of this directive. Moreover, exempting the Coast Guard and the 
Secret Service from the provisions of this directive casts doubt on the 
authority and accountability relationships between these components and 
the CHCO, potentially complicating the department's objective of 
clarifying the responsibilities, accountability, and authorities of the CHCO 
office and the components set forth in DHS's training strategic plan. 



While several provisions in the Homeland Security Act require the Coast Guard to be 
maintained as a distinct entity and would limit the range of management initiatives 
regarding the Coast Guard, none of them would appear to be applicable in this case. We 
find nothing in the DHS management directive on the integration of human capital that 
contravenes these limitations and nothing in the directive would reasonably appear to 
threaten the status of the Coast Guard as a distinct entity or otherwise impair its ability to 
perform statutory missions. We have reported on a similar situation with respect to the 
department's acquisition function. See GAO, Homeland Security: Successes and 
Challenges in DHS's Efforts to Create an Effective Acquisition Organization, GAO-05-179 
(Washington, D.C.: Mar. 29, 2005). 

20 Similarly, DHS's management directive on human capital integration also asserts that the 
Secret Service is exempted by statute. As with the Coast Guard, we are unaware of any 
specific statutory exemption that would prevent the application of the DHS management 
directive regarding human capital, and given the nature of the management directive, we do 
not see any apparent reason to exempt the Secret Service. 
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Planning May Be Insufficiently m an d of itself, DHS's dual accountability authority structure is not an 
Detailed to Ensure Effective obstacle to implementation of departmentwide training efforts. However, 
Implementation in "Dual without detailed implementation plans, it presents potentially significant 

Accountability" Environment challenges. Because of this shared authority, DHS will need to take great 

care when planning for departmentwide training initiatives involving 
multiple organizational components to ensure that resources are aligned 
with organizational units performing activities, especially related to cross- 
organizational sharing of training and delivery of common training. The 
lack of comprehensive and rigorous planning can lead to confusion over 
responsibilities, lack of coordination, and missed deadlines. Regular and 
rigorous use of detailed implementation plans is necessary to implement 
decisions and carry out activities in a coordinated manner. 

After we completed our audit work, DHS informed us that it plans to 
establish 31 tactic teams to take ownership of each of the tactics 
presented in the DHS training strategic plan to be completed by the end of 
fiscal year 2006. As of mid-August 2005, DHS provided us with 
documentation indicating that 3 of these teams have been established to 
date. These teams appear to have taken promising steps toward the 
establishment of detailed plans for implementing their respective training 
tactics by developing draft objectives, deliverables, and closure criteria. 
But as fiscal year 2006 approaches, time is short for the CHCO office and 
the components to establish the remaining teams and then take the actions 
necessary to develop and put in place the detailed plans that will be 
critical for effectively implementing DHS's many training tactics by the 
end of the coming fiscal year. The TLC's ADLG has made use of this type 
of detailed approach in a report proposing a distance learning architecture 
for the department. Appended to its report is a detailed plan outlining the 
major activities, milestones, resources, and components needed to support 
the successful implementation of the proposal. 

Resources Needed to Several training officials told us they were concerned about the lack of 

Implement Departmental dedicated resources and related capacity to carry out departmental 

Training Strategy initiatives. At the time we started our review, the CHCO office had only 

one full-time permanent employee dedicated to carrying out these 
activities; consequently, both training leaders and staff from organizational 
components were relied on to contribute to departmentwide efforts. After 
we concluded our audit work, a DHS official told us that the CHCO office 
had recently hired two additional full-time training staff: an ADL program 
manager and a staffer to oversee a recently approved SES candidate 
development program and headquarters operational leadership 
development. Individual components have also provided some assistance 
to departmentwide efforts through the appointment of temporary 
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personnel. In late 2004, CBP and FLETC each detailed a staff member to 
the CHCO office to work on training-related projects. In addition, DHS has 
contracted for services to address selected departmentwide issues, such 
as setting common standards for ADL and reviewing DHS training 
facilities. 

DHS's departmental training councils and groups are almost exclusively 
staffed by component training leaders who already have full-time training 
commitments. The department's training strategic plan identifies many 
tactics for fiscal year 2006 — including creating a common training 
language and glossary of training terms, establishing a repository for 
course catalog information, and developing a DHS training Web site — that 
will require considerable staff support to implement. Successful and timely 
completion of these and other initiatives will depend on sufficient 
resources being provided. 



It is essential for federal agencies to ensure that their training efforts are 
part of — and are driven by — their organizational strategic and 
performance planning processes. We have reported that aligning training 
with strategic priorities and systematically evaluating training activities 
play key roles in helping agencies to ensure that training is strategically 
focused on improving performance and meeting overall organizational 
goals. 21 Strategic training practices in several DHS components or 
programs may provide models or insights to others in the department 
regarding ways to improve training practices. 22 In areas where some 
components employed strategic practices, other components did not. 



We have previously reported that agencies demonstrating a strategic 
approach to training align their training efforts with overall strategic 
priorities. To do this, agencies can employ a variety of practices, such as 
linking training activities to strategic planning and budgeting and 
performing front-end analysis to ensure that training activities are not 



21 GAO-04-546G. 

22 We have previously reported that organizations undergoing successful transformations 
look for and implement best practices wherever they may be found. See GAO-03-669 and 
GAO, Highlights of a GAO Forum, Mergers and Transformation: Lessons Learned for a 
Department of Homeland Security and Other Federal Agencies, GAO-03-293SP 
(Washington, D.C.: Nov. 14, 2002). 



DHS's Approach to 
Planning and 
Evaluation of Training 
Demonstrates Some 
Elements of Strategic 
Practice, but Is Still 
Evolving 



Aligning Training with 
Organizational Priorities Is 
a Strategic Practice 
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initiated in an ad hoc, uncoordinated manner, but rather are focused on 
improving performance toward the agency's goals. 23 Some components in 
DHS applied the strategic practice of aligning training with organizational 
priorities, while others did not. 

CBP links its new and existing training activities to its strategic priorities 
when planning for its strategic initiatives and expenditures. Importantly, 
the head of training at CBP is at the decision-making table with other CBP 
leaders to help establish training priorities consistent with the priorities of 
the CBP Commissioner. Relevant program managers are asked, "What 
training do you need to achieve the goals in your strategic plan?" Such 
discussions took place during planning for CBP's custom trade pact 
initiative. 

During each budget cycle, CBP's central training office issues a "call for 
training" to its mission and mission support customers to estimate CBP's 
training needs for existing training activities and prioritize these needs 
based upon the Commissioner's priorities. Prior to establishing this 
process, training was mostly decided on a first come, first serve basis 
without clear and transparent linkages to organizational priorities. CBP's 
current process results in an annual training plan in which training needs 
are identified by priority as well as major occupational type, such as 
border patrol agent or CBP officer. Training decisions are based on 
whether training requested is critical, necessary, or "nice to have." During 
fiscal year training plan implementation, CBP tracks actual training 
activity through a central database to determine whether CBP is using its 
planned training resources. By tracking plan usage through a centrally 
managed database, CBP is able to reallocate unused training funds prior to 
the end of the fiscal year for either training activities that were not 
included in its original plan because of capacity constraints, or for 
emerging training priorities. 

The Coast Guard has adopted a strategic and analytic approach to training 
through its use of the Human Performance Technology (HPT) model — a 
front-end training assessment process to determine the cause of 
performance problems. The process starts with the assumption that many 
factors influence individual and unit performance and it is important to 
determine what the factors are before concluding that training is the 
solution. From its HPT analysis, the Coast Guard determines whether 



" GAO-04-546G. 
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training is needed or whether another type of solution, such as a policy 
change, would be more appropriate. For example, in addressing a problem 
in aviation maintenance, a Coast Guard working group looked at likely 
causes of its performance problems and concluded that focusing on 
making aviation maintenance training better was not the only solution. 
More specifically, training officials encountered problems with job 
dissatisfaction and subpar performance from aviation chief warrant 
officers. In this case, training officials used HPT to analyze the nature of 
work performed by those responsible for aviation maintenance and 
concluded that there was not a good match between job skills and 
responsibilities. Specifically, over the last 20 years, the scope and nature of 
the work performed by chief warrant officers changed significantly from 
maintaining components to managing aircraft systems. Performance 
problems were mainly caused by significant changes in the job functions 
of these officers over the years rather than by a lack of adequate training. 

In cases where the HPT analysis concludes that training is warranted, a 
training analysis is performed to determine the specific training 
interventions. For example, in implementing activities related to the 
Maritime Transportation Security Act, the Coast Guard analyzed its 
training needs through the HPT process to determine training necessary to 
help maritime inspectors reduce the exposure of ports and waterways to 
terrorist activities. The analysis identified the skills and knowledge 
necessary for new maritime inspector tasks and provided training 
interventions, such as developing job aids and targeted classes, to prepare 
inspectors for the tasks most relevant to support their new role. New 
courses were piloted and then subjected to multilevel evaluations to 
assess their effectiveness and potential impact on employee performance. 



Systematic Evaluation of Agencies demonstrating a strategic approach to training employ a variety 
Training Is a Strategic °f practices, such as systematically evaluating training, actively 

Practice incorporating feedback during training design, and using feedback from 

multiple perspectives. 24 Several components and programs we examined at 
DHS used these practices, while others did not. 
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One commonly accepted model used for assessing and evaluating training 
programs consists of five levels of assessment (see fig. 4). 25 In our review, 
virtually all components captured Level I data focusing on end-of-course 
reactions, while several also collected Level II data focusing on changes in 
employee skill, knowledge, or abilities. Several components evaluated, or 
were planning to evaluate, the impact of selected training programs on 
individual behavior, represented by Level III evaluations. 



Figure 4: Levels of Training Evaluation 



How does the monetary value of 
results of the program compare with 
the related costs? 



Did the learner apply the new skills 
to the necessary tasks in the 
organization? If so, what results 
were achieved? 



What skills did the learner develop? 
What new information is the learner 
using on the job? 

What facts or knowledge did the 
learner gain? 

How does the learner feel about 
the training? 



Source: GAO (presentation). 

Note: GAO presentation based on information from Donald L. Kirkpatrick, Evaluating Training 
Programs: The Four Levels (San Francisco: Berrett-Koehler Publishers, Inc., 1998), and Jack J. 
Phillips, ed., Implementing Evaluation Systems and Processes (Alexandria, Va.: American Society for 
Training and Development, 1998). 

To measure the real impact of training, however, agencies need to move 
beyond data focused primarily on inputs and outputs and develop 
additional indicators that help determine how training efforts contribute to 



25 Donald L. Kirkpatrick, author of Evaluating Training Programs: The Four Levels (San 
Francisco: Berrett-Koehler Publishers, Inc., 1998), developed a commonly recognized four- 
level model for evaluating training and development efforts. The fourth level is sometimes 
split into two levels with the fifth level, return on investment, representing a comparison of 
costs and benefits quantified in dollars. See Jack J. Philips, Implementing Evaluation 
Systems and Processes. 
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the accomplishment of agency goals and objectives. At a couple of 
components, DHS officials told us they conducted Level IV evaluations, 
which assess the effectiveness of training interventions. We found no 
examples of the department or its components measuring the return on 
investment of training activities (Level V). 26 Training effectiveness should 
be measured against organizational performance; however, not all levels of 
training evaluation require or are suitable for return on investment 
analysis. Determining whether training programs merit the cost of using 
such an approach depends upon the programs' significance and 
appropriateness. 

CBP takes a systematic approach to evaluating its training activities 
through its National Training Evaluation Program (NTEP) to help program 
managers and trainers make more informed decisions on the effectiveness 
of training courses and their delivery. Despite the fact that CBP is a large 
and decentralized organization, NTEP has enabled it to collect course 
evaluation information and make this information available to a wide 
range of users in a timely manner. NTEP has also standardized evaluation 
data to allow for comparison of training throughout various field locations. 
Before the rollout of NTEP, CBP did not use a standard mechanism for 
collecting evaluation data, which, according to a CBP official, made it 
difficult to gather evaluation data nationally. 

CBP focuses on collecting both end-of-course student reactions (Level I) 
and supervisor assessments of student on-the-job performance after 
attending the training (Level III). Electronic or paper-based evaluations 
are entered into the NTEP information system. The "close to real time" 
online data enables supervisors to perform trend analysis on training 
quality and provides opportunities for them to troubleshoot training 
deficiencies and identify high-performing courses. The NTEP online 
system allows CBP employees access to evaluation data on a need to 



Higher levels of evaluation, and in particular Level V, can be challenging to conduct 
because of the difficulty and costs associated with data collection and the complexity in 
directly linking training programs to improved individual and organizational performance. 
Factors to consider when deciding the appropriate level of evaluation include estimated 
costs of the training effort, size of the training audience, management interest, program 
visibility, and the anticipated "life span" of the effort. In light of these considerations, an 
agency may decide to evaluate participants' reactions (Level I) for all of its training 
programs, while conducting a return on investment analysis (Level V) for only a very few. 
Each agency will need to consider the feasibility and cost-effectiveness of conducting these 
in-depth evaluations, along with budgetary and staffing circumstances that may limit the 
agency's ability to complete such evaluations. 
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know basis with four levels of access, while enabling them to locate 
evaluation data for any training class by date. Evaluation reports are 
aggregated for review by senior CBP officials. 

A CBP official told us that collecting course evaluation data is labor 
intensive, especially since many field operations still use paper processes. 
In addition, CBP has experienced a relatively low submission rate for 
Level I evaluation data for many of its training classes. The official told us 
that this was especially true for end-of-course reactions from staff in the 
field, where only about one-third of officer-related course participants 
submit evaluation forms. Given cost and labor challenges, CBP has 
targeted areas for evaluation that it believes are important, such as 
training related to its "One Face at the Border" initiative. 

In addition, agencies with a strategic approach to training do not wait until 
the conclusion of a training intervention to conduct evaluations. Rather, 
they approach evaluation through an iterative process capable of 
informing all stages of training. 27 DHS's CHCO office used multiple forms 
of feedback from employees to develop its training strategy for MAX™. 
From February through April 2005, the department administered surveys 
and conducted focus groups to obtain information on the needs, attitudes, 
and reactions of different communities affected by MAX™. Shortly after 
issuing its new human capital regulations, the department provided basic 
information to all employees on the nature and timeline of changes they 
could expect under MAX™ through a Web broadcast. After the broadcast 
an online survey was used to obtain feedback from employees regarding 
the broadcast itself and their general feelings and concerns about the 
MAX™ initiative. 

DHS followed this initial survey with a larger survey to gather additional 
feedback on how information regarding MAX™ had been communicated, 
as well as specific areas where employees wanted additional information. 
Concerns about the need for training were prominent among the more 
than 9,000 responses received, with respondents ranking training as the 
second most serious challenge to the successful implementation of MAX™. 
According to a senior DHS official, the survey results will inform 
subsequent training and communication efforts. 
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DHS also collected evaluative feedback by conducting a series of focus 
groups held in locations across the country. The aim of these sessions was 
to validate the design of the performance management program 
established under MAX™ and identify concerns that would inform the 
development of additional training. Consistent with the strategic training 
practice of seeking out different perspectives when redesigning and 
assessing training efforts, DHS staff held separate focus groups for 
bargaining unit employees, non-bargaining unit employees, and 
supervisors and managers at all of the locations visited. This enabled them 
to identify issues of particular concern to each of these groups as well as 
issues common to all three. For example, both the bargaining unit and 
non-bargaining unit employee focus groups raised concerns about 
supervisors having inadequate skills for fairly administering the new 
performance management system. This concern was also shared by 
supervisors and managers themselves who expressed the need for 
additional skills training in areas such as goal setting and providing 
performance feedback. The sessions validated the CHCO office's plans to 
offer performance management training to supervisors and managers 
before the implementation of the new system and assisted in refining 
issues for future training. 

FLETC's methods for evaluating its major training programs include 
feedback from multiple perspectives when examining the benefits of 
training on actual employee job performance. FLETC's Level III 
evaluations obtain feedback from both trainees and their supervisors to 
inform future improvements to training curricula. Evaluation results are 
compiled into a comprehensive report used during FLETC's periodic 
curriculum reviews on its major training programs, such as the Natural 
Resource Police Training Program. The report contains detailed feedback 
from both the trainee and supervisor perspectives 6 months to 1 year after 
the trainee has attended the training program. For example, for the 
Natural Resource Police Training Program, FLETC analyzed how well the 
program prepared trainees in all aspects of their jobs. In this case, analysis 
identified those courses that had benefited program trainees the least — 
including determining speed from skid marks and death notification. 
Training designers can use report information to improve program 
curricula and refocus training on knowledge and skill areas most critical 
to performing the job. In addition to Level III evaluation results, its 
training designers make program and individual class changes by using 
other methods of evaluation, such as direct student feedback after classes 
and trainee examinations, which determine how well the trainees 
understood the course material immediately following the program. 
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DHS Has Used 
Training in an Effort 
to Help Its Workforce 
as the Department 
Undergoes 
Transformation and 
Cultural Change 



The creation of DHS resulted in significant cultural and transformational 
challenges for the department. We have previously reported that training is 
one way organizations successfully address cultural issues while 
simultaneously facilitating new ways to work toward the achievement of 
organizational goals. 28 Among the DHS components in our review, some 
merged cultures from different legacy organizations (CBP, ICE), another 
component came as a small organization that greatly expanded when 
joining DHS (Federal Air Marshal Service), while others joined DHS intact 
(Secret Service, Coast Guard, FLETC), and still another was previously a 
part of a larger legacy organization (CIS). Each component faces the need 
to find a way to identify itself as part of the larger DHS organization, that 
is, with a sense of affiliation rather than as an outsider looking in. At the 
same time, components must either maintain their existing cultures or 
develop new cultures to adapt to changing missions and needs. The key is 
to build upon positive aspects of the components' cultures as the larger 
organization develops its own culture. 



Agencies that undergo successful transformations change more than just 
their organizational charts, they also make fundamental changes in basic 
operations, such as how they approach strategic human capital 
management. DHS understands this, and the MAX™ initiative is part of an 
effort by the department to fundamentally change its approach to human 
capital management by establishing a personnel system that is flexible, 
performance oriented, and market based. 



The Secretary of Homeland Security and other top officials have actively 
supported the role of training in implementing these changes by making it 
a leadership expectation that all DHS executives, managers, and 
supervisors be personally involved as both participants in and supporters 
of MAX™ training efforts. The CHCO office, working with the assistance of 



See GAO, Organizational Culture: Techniques Companies Use to Perpetuate or Change 
Beliefs and Values, GAO/NSIAD-92-105 (Washington, D.C.: Feb. 27, 1992), and 
Organizational Culture: Use of Training to Help Change DOD Inventory Management 
Culture, GAO/NSIAD-94-193 (Washington, D.C.: Aug. 30, 1994). The first of these reports 
examined techniques used by nine large private sector firms to affect their organizational 
cultures. We found that company officials identified two techniques that were of very great 
importance to a successful culture change: (1) total commitment of top management and 
(2) training that promotes and develops skills in line with the desired culture. More 
recently, we have reported on the cultural changes and key practices necessary for 
successful transformation. See, GAO, 21st Century Challenges: Transforming 
Government to Meet Current and Emerging Challenges, GAO-05-830T (Washington, D.C.: 
July 13, 2005). 
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outside contractors, has developed several training interventions aimed at 
providing these groups with the tools and information needed to champion 
the benefits of a performance-based culture and successfully implement 
MAX™ in their components. 

In August 2005, DHS sponsored a day training program for 350 to 400 
of the department's senior executives and flag officers. The program 
covered a range of topics, including an update on current DHS priorities; 
techniques and best practices for how senior leaders can effectively 
support and implement these priorities; as well as specific management, 
communication, and training approaches that can be used to support the 
creation of a performance-based culture. The Secretary, Deputy Secretary, 
and Under Secretary for Management all participated in the program, 
which also featured presentations from human capital and organizational 
change experts from outside the department. In addition to its focus on 
MAX™ implementation, which included both large and small group 
sessions wherein participants could discuss performance management and 
share information on practices, the course also provided a forum for the 
department's top leadership and senior executives to review the then 
recently issued recommendations resulting from the Secretary's Second 
Stage Review process. 

Another training intervention sponsored by the department directly targets 
managers and supervisors who will be responsible for carrying out many 
of the key behaviors associated with the new system and whose active 
support is viewed by DHS as critical for achieving the transformation to a 
performance-based culture. The 2-V6 day program focuses on developing 
and improving interpersonal, managerial, and other so-called soft skills. 
DHS expects to provide the training to approximately 12,000 managers and 
supervisors throughout the department. 

On the component level, training has also played an important role in 
CBP's effort to transform from the traditional, largely siloed approach 
used by its legacy agencies when protecting our borders to a new 
integrated concept that it believes is more in line with its current needs. 
Officials noted that the merger into CBP led to some resistance from 
employees who had not yet understood or accepted the reasons for the 
merger. These same officials acknowledged that they must continue to 
work at informing employees why changes were made and provide 
vehicles for better integration through training. For example, in the "One 
Face at the Border" initiative, supervisory training has incorporated some 
elements of cultural integration by including a session on bridging the 
culture gap. Officials at CBP designed and piloted a training module to be 
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added to the supervisory curriculum specifically targeting how they can 
more effectively understand the value and perspective of staff coming out 
of the legacy organizational cultures. 

In addition, training played a key role in facilitating the transition of CBP's 
workforce from its three legacy organizations. Training for the new CBP 
officer and CBP agriculture specialist positions aimed to improve 
coordination and communication across inspection functions and enhance 
the flexibility of CBP's workforce. Specifically, CBP created a series of 
training courses to provide former Customs and former Immigration and 
Naturalization Service officers with the knowledge and skills necessary to 
carry out the responsibilities of this new position. To develop this training, 
CBP-wide working groups identified and validated critical tasks for the 
new frontline CBP officer to perform. A mix of training delivery methods 
were used (i.e., e-learning and instructor led), and classroom knowledge 
and skills were reinforced with on-the-job training. CBP provided 
extensive train-the-trainer courses so that trainers could return to their 
field sites and instruct officers there. (See app. 2.) 



DHS must continue to make progress on three important aspects of 
training as it moves forward: (1) forging an effective role for training at the 
departmental level and implementing its departmentwide training strategy; 
(2) taking a strategic approach to training practices, in part by building 
upon examples of good practice to be found among its former 
organizations, as well as considering other examples of strategic practices; 
and (3) finding ways that training can help to foster organizational 
transformation and cultural change within the department. To date, DHS 
has taken positive steps in these areas, yet significant challenges lie ahead. 

The ability to make decisions from a departmentwide perspective and then 
effectively implement them will help determine whether training in DHS 
achieves its intended results. Strong leadership will play a critical role in 
this process. To be successful, DHS will need to have both a clear plan of 
action as well as the ability to anticipate and overcome several 
implementation challenges. The creation of the TLC and the development 
of the department's first training strategic plan both represent a good start 
in this process. Better performance measures, more specific milestones, 
and the inclusion of performance targets would make DHS's strategic 
training plan a more useful tool for both internal and external stakeholders 
to use in tracking the department's progress toward achieving its training 
objectives. Clarifying authority relationships between the CHCO and 
component heads, developing detailed implementation plans, and giving 
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appropriate attention to providing resources to implement training 
initiatives when setting funding priorities are also likely to be critical 
factors in building and sustaining an effective role for department-level 
training at DHS. 

A strategic approach toward training is also very important as DHS seeks 
to build on its current efforts and strives to move forward. As we have 
noted, some programs and components in DHS already use specific 
strategic training practices, and other components within the department 
can benefit from their example. As DHS implements new training 
programs, such as the large-scale, multistage training being developed to 
support the implementation of MAX™, it has a valuable opportunity to 
reflect the lessons learned from these experiences in subsequent 
departmentwide training efforts. Finally, the transition to a new 
department has brought with it cultural challenges, and training can play a 
role in both defining and refining an effective DHS culture without 
sacrificing the cultural history of its components. 



RGCOrQmcndcltionS for ^° es ^ a " :> ^ sn anc ^ implement an effective and strategic approach 

to departmentwide training, we recommend that the Secretary of 
Executive Action Homeland Security take the following actions: 

• adopt additional good strategic planning and management practices to 
enhance the department's training strategic plan by (1) creating a clearer 
crosswalk between specific training goals and objectives and DHS's 
organizational and human capital strategic goals and (2) developing 
appropriate training performance measures and targets; 

• clearly specify authority and accountability relationships between the 
CHCO office and organizational components regarding training as a first 
step to addressing issues DHS has identified for fiscal year 2006; 

• ensure that the department and component organizations develop detailed 
implementation plans and related processes for training initiatives; and 

• when setting funding priorities, give appropriate attention to providing 
resources to support training councils and groups to further DHS's 
capacity to achieve its departmentwide training goals. 



We provided a draft of this report to the Secretary of Homeland Security 
for comment and received written comments from DHS that are reprinted 
in appendix III. In addition, we received technical comments and 
clarifications, which we incorporated where appropriate. DHS generally 
agreed with our recommendations. 



Agency Comments 
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We will provide copies of this report to the Secretary of Homeland 
Security and other interested parties. Copies will also be provided to 
others upon request. In addition, this report is available at no charge on 
the GAO Web site at http://www.gao.gov. 

If you have any questions about this report, please contact me at (202) 512- 
9490 or stalcupg@gao.gov. Major contributors to this report were Kimberly 
Gianopoulos, Assistant Director; Peter J. Del Toro; Robert Yetvin; and 
Gerard Burke. 

Sincerely yours, 




George H. Stalcup 
Director, Strategic Issues 
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Appendix I: Scope and Methodology 



To achieve our objectives, we reviewed training at the Department of 
Homeland Security (DHS) at both the departmental and component levels. 
When examining training at the departmental level, we collected, 
reviewed, and analyzed the department's training rules, procedures, 
policies, and organizational charts; departmental, human capital, and 
training strategic plans; human capital and training management 
directives; Internet and intranet Web pages; and other relevant documents. 
To further our understanding of training at DHS and the issues and 
challenges involved, we interviewed training and human capital officials in 
the Office of the Chief Human Capital Officer and the leaders and 
coleaders of DHS's training councils and groups. We also observed the 
January 2005 meeting of the Training Leaders Council. We supplemented 
our review of departmental training at DHS by examining the department's 
effort to use training related to MAX™ to foster transformation and 
cultural change in the department. 

In addition, we reviewed training at major organizational components in 
DHS and selected the six largest components based on staff size and 
budget. Using these criteria, we reviewed training at Customs and Border 
Protection (CBP), Citizenship and Immigration Services, Immigration and 
Customs Enforcement (including the Federal Air Marshal Service, the 
Federal Protective Service, and the Leadership Development Center), the 
Coast Guard, the Secret Service, and the Transportation Security 
Administration. See figure 5 for a depiction of the DHS organizational 
structure in place during the time of our review. These components 
collectively represent about 95 percent of the total staff at DHS. We also 
included the Federal Law Enforcement Training Center because of the 
special role it plays in training employees from other DHS components. 
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Appendix I: Scope and Methodology 



Figure 5: DHS Components Included in Our Review 
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Source: GAO representation of DHS information. 

Note: This figure does not depict all components and entities in the department, but is intended to 
provide a general framework within which to present the components and centers included in our 
review. The organizational structure depicted above was in place during our review, and does not 
reflect actual or proposed changes related to DHS's Second Stage Review. 



When examining training at selected components, we reviewed 
component-level strategic, human capital, and training plans when 
available; training budget requests and expenditure documents; training 
procedures, policies, and organizational charts; rules and policies for 
identifying and prioritizing training programs; Internet and intranet Web 
pages; selected training course materials; and other relevant documents 
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Appendix I: Scope and Methodology 



produced by these components. To further our understanding of training 
at the component level, we also interviewed training officials at each of 
the selected components and identified these individuals based on their 
knowledge, experience, and leadership roles. We conducted our 
interviews at component headquarters or field offices located in the 
Washington, D.C., area. In addition, as part of our review of DHS's efforts 
to foster transformation and cultural change, we observed training related 
to CBP's "One Face at the Border" initiative in northern Virginia. 

To help determine whether DHS used a strategic approach in planning and 
evaluating its training activities at the departmental or component levels, 
we referenced criteria contained in our guide for assessing strategic 
training and development efforts in the federal government. 1 This guide 
outlines a framework for assessing training efforts, consisting of a set of 
principles and key questions that federal agencies can use to ensure that 
their training investments are targeted strategically and not wasted on 
efforts that are irrelevant, duplicative, or ineffective. We selected our case 
examples based on their suitability for demonstrating specific strategic 
training practices. Other components within DHS may, or may not, be 
engaged in similar practices. To determine whether DHS followed leading 
management practices in planning and implementing departmentwide 
training, we also drew on our previous work on strategic planning and 
effective management practices. 

We did not include within our scope training intended for audiences 
external to DHS, and we generally covered training and training 
management in effect during the period in which we did our work. We 
conducted our work from November 2004 through July 2005 in accordance 
with generally accepted government auditing standards. We requested 
comments on a draft of this report from DHS, which are reprinted in 
appendix III. The comments are addressed in the Agency Comments 
section of this report. 



1 GAO-04-546G. 
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Appendix II: "One Face at the Border" 
Training at U.S. Customs and Border 
Protection 



One of the initial goals for creating DHS was to better protect the United 
States from terrorists entering the country, and ports of entry are the 
means through which terrorists can enter. The creation of CBP within 
DHS merged border inspection functions at U.S. ports of entry, which had 
previously been performed by three separate agencies. Known as "One 
Face at the Border," this initiative created the positions of CBP officer and 
CBP agriculture specialist that combined aspects of three former inspector 
functions. This initiative aimed to improve coordination and 
communication of inspections to better protect the nation's borders from 
terrorists as well as to improve entry for legitimate travel and trade. 

To successfully make the transition to these new positions, significant 
training was needed. Specifically, CBP created a series of training courses 
to provide former U.S. Customs and former Immigration and 
Naturalization Service officers with the knowledge and skills necessary to 
carry out the responsibilities of this new position. In addition, CBP officers 
received training to meet CBP's new mission priority of terrorism 
prevention. Although the emphasis was on cross-training legacy officers, 
the new curriculum was also geared to new hires. Because agricultural 
inspections are more specialized, CBP officers receive training sufficient 
to enable them to identify potential agricultural threats, make initial 
regulatory decisions, and determine when to make referrals to CBP 
agriculture specialists. More detailed agricultural inspections are 
performed by these specialists who have substantial training and 
background in agricultural issues. 

A variety of training delivery methods were used (e.g., e-learning and 
classroom) and these training methods were reinforced with extensive on- 
the-job training. In addition to traditional content areas (e.g., cross-training 
for former U.S. Customs officers includes courses on immigration 
fundamentals and immigration law), training courses also covered CBP's 
new priority mission of preventing terrorism (e.g., training in detecting 
possible terrorists and fraudulent documents, honing interviewing skills, 
and making appropriate referrals to staff for additional inspection). CBP 
emphasizes on-the-job training in an effort not to place inspectors on the 
job without direct supervisory and tutorial backup. Training for new 
recruits has also been modified to include a preacademy orientation 
program at the port location where the recruit will eventually work before 
he or she receives academy training. This is a 72-day course for CBP 
officers and a 46-day course for CBP agriculture specialists. 

CBP's main strategy to prepare for field delivery of training was to provide 
extensive train-the-trainer courses so that trainers could return to their 
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Appendix II: "One Face at the 
Border" Training at U.S. 
Customs and Border Protection 



field sites and instruct officers there. Training priorities were established 
with the idea of spacing the training out so that field offices would not be 
overwhelmed. For example, CBP rolled out its primary cross-training to 
airports, while antiterrorism training was rolled out to land borders. 

Officials reported that cross-training benefited CBP officers since they 
have gained more knowledge by learning both immigration and customs 
laws and procedures. This increase in knowledge has the potential benefit 
of providing more variety in job tasks as well as increasing the 
opportunities for advancement since an officer can now apply for 
supervisory-level positions that had previously been open only to former 
U.S. Customs or Immigration and Naturalization Service officers. 

Change has not come about without challenges, however, as many officers 
were reported to have resisted changes to their responsibilities, mainly 
related to the difficulties in learning a new set of procedures and laws. 
Officials noted that there has been an enormous amount of required 
training for CBP officers, and it can sometimes be overwhelming. For 
former officers, in addition to completing an extensive cross-training 
schedule and new training related to terrorism prevention, there are many 
other required courses related to their mission. For example, training 
modules are required in areas such as body scanning, hazardous materials, 
cargo inspection, and seized assets. 

Although staffing challenges may ultimately be relieved with trained 
officers able to perform dual inspections, officials noted that it has been 
extremely difficult to take staff off-line to complete the "One Face at the 
Border" training. One official said that classes have been very difficult to 
schedule because of the constant pressure to staff operations. For 
example, in one case, a class was canceled right after it began because the 
trainees were pulled out to staff their inspection booths. This official also 
noted that trainers have had to be very flexible to accommodate staff 
schedules to ensure that training occurs. 



Page 44 



GAO-05-888 Training at DHS 



Appendix III: Comments from the 
Department of Homeland Security 



U.S. Department of Homeland Security 

Washington, DC 20528 



Homeland 
l l§F Security 



September 7, 2005 



Mr. George H. Stalcup 

Director 

Strategic Issues 

U.S. Government Accountability Office (GAO) 
441 G Street, NW, 2440C 
Washington, DC 20548 

Dear Mr. Stalcup: 

RE: GAO-05-888, Department of Homeland Security (DHS): Strategic Management of Training 
Important for Successful Transformation (GAO Job Code 450353). 

Thank you for the opportunity to review the draft report. The report is a positive statement of the 
initial actions DHS has taken to ensure that training is used as a resource for transformation of the 
Department. We concur that the creation of the Training Leaders Council (TLC) and the 
development of the initial DHS Learning and Development Strategic Plan, "is a significant and 
positive step towards addressing department wide training challenges." ^Further, we agree that 
implementation of specific training programs, such as training for MAX , is important to our 
efforts to meet our goal for organizational excellence. 

The Chief Human Capital Office was pleased to provide the GAO with reports about our training 
organizations and to coordinate interviews with DHS training leaders. Although, the training 
enterprise within the Department is a major undertaking with limited resources, we believe that 
external review and input provides us the opportunity for improvement. We encourage GAO to use 
the information gathered while creating this report as a baseline to streamline the process for future 
reviews. Technical and substantive comments were provided separately for your use in updating the 
final report. 

We concur with the intent of the recommendations in the report and have already taken steps to 
ensure that measures are appropriately used when implementing the Learning and Development 
Strategic Plan. The TLC designed the plan to link to the President's Management Agenda and to the 
DHS Strategic Plan, but will take further steps to make those links more transparent to external 
audiences. We anticipate that the authority and the role of the CHCO will be further clarified as we 
work towards functional integration and as the TLC develops a full accounting of current training 
authorities and requirements. Our plan is to ensure that component organizations complete annual 
training plans by March, 2006. We recognize that additional resources would enhance the TLC's 
ability to implement training strategies but will need to balance that issue with on-going mission 
requirements. 



www.dhs.gov 
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of Homeland Security 



We look forward to the final report and plan to provide additional comments at that time regarding 
the substance and recommendations contained in the report. 



Sincerely, 

Steven J. Pecinovsky 
Director 

Departmental GAO Audit Liaison Office 




(450353) 
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GAO's Mission ^ ne Government Accountability Office, the audit, evaluation and 

investigative arm of Congress, exists to support Congress in meeting its 
constitutional responsibilities and to help improve the performance and 
accountability of the federal government for the American people. GAO 
examines the use of public funds; evaluates federal programs and policies; 
and provides analyses, recommendations, and other assistance to help 
Congress make informed oversight, policy, and funding decisions. GAO's 
commitment to good government is reflected in its core values of 
accountability, integrity, and reliability. 



Obtaining CODiGS Of ^ e ^ as * es ^ anc ^ eas i es t wa y t° obtain copies of GAO documents at no cost 

o tr i s through GAO's Web site (www.gao.gov). Each weekday, GAO posts 

GAO Reports and. newly released reports, testimony, and correspondence on its Web site. To 

TfiQti m nnv nave GAO e-mail you a list of newly posted products every afternoon, go 
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Order by Mail or Phone The first copy of each printed report is free. Additional copies are $2 each. 

A check or money order should be made out to the Superintendent of 
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more copies mailed to a single address are discounted 25 percent. Orders 
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U.S. Government Accountability Office 
441 G Street NW, Room LM 
Washington, D.C. 20548 

To order by Phone: Voice: (202)512-6000 
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Fax: (202) 512-6061 



Contact: 



To Report Fraud, 
Waste and Abuse in s ^ e: w ^^ g a ° s° v /f rau(me ^f rau dnet.htm 

' E-mail: fraudnet@gao.gov 

Federal Programs Automated answering system: (800) 424-5454 or (202) 512-7470 



P on 0rP«s«si on a 1 Gloria Jarmon, Managing Director, JarmonG@gao.gov (202) 512-4400 

V^Ul Lgl es&lUl Ldl u.S. Government Accountability Office, 441 G Street NW, Room 7125 

Relations Washington, D.C. 20548 



Pllhli C Affairs ^ au ^ Anderson, Managing Director, AndersonPl@gao.gov (202) 512-4800 

r ^lldll & u.S. Government Accountability Office, 441 G Street NW, Room 7149 

Washington, D.C. 20548 



PRINTED ON T^£} RECYCLED PAPER 



